2026-04-10 18:30:30,500 [ 189532 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 18:30:30,500 [ 189532 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 18:30:30,500 [ 189532 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 18:30:30,500 [ 189532 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_xo6f5c --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2]' test_executable_user_defined_functions_config_reload/test.py::test test_explain_estimates/test.py::test_explain_estimates test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node]' 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path]' test_filesystem_cache/test.py::test_custom_cached_disk test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio 'test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node]' test_filesystem_cache/test.py::test_system_sync_filesystem_cache test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_http_and_readonly/test.py::test_http_get_is_readonly test_http_native/test.py::test_http_native_returns_timezone test_inherit_multiple_profiles/test.py::test_combined_profile test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_jbod_ha/test.py::test_jbod_ha test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin test_keeper_block_acl/test.py::test_keeper_block_acl test_keeper_compression/test_with_compression.py::test_select test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks test_keeper_four_word_command/test_allow_list.py::test_allow_list test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password test_keeper_memory_soft_limit/test.py::test_soft_limit_create test_keeper_mntr_pressure/test.py::test_aggressive_mntr test_keeper_nodes_move/test.py::test_node_move test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout 'test_keeper_zookeeper_converter/test.py::test_acls[False]' 'test_keeper_zookeeper_converter/test.py::test_acls[True]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_smoke[False]' 'test_keeper_zookeeper_converter/test.py::test_smoke[True]' test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain]' test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_reload_clusters_config/test.py::test_add_cluster test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_mask_sensitive_info/test.py::test_backup_table test_keeper_zookeeper_converter/test.py::test_acls[False] test_external_http_authenticator/test.py::test_basic_auth_failed test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] [gw5] [ 1%] PASSED test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed [gw5] [ 2%] PASSED test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response [gw7] [ 3%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes [gw7] [ 4%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk [gw6] [ 5%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] [gw9] [ 6%] PASSED test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard [gw6] [ 7%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] test_insert_into_distributed_sync_async/test.py::test_insertion_sync [gw6] [ 8%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] [gw6] [ 9%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] [gw2] [ 10%] PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] [gw6] [ 11%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] test_keeper_persistent_log/test.py::test_ephemeral_after_restart [gw7] [ 12%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin [gw5] [ 13%] PASSED test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass [gw9] [ 14%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout [gw2] [ 15%] PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] test_filesystem_cache/test.py::test_custom_cached_disk [gw7] [ 16%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin [gw5] [ 17%] PASSED test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass [gw5] [ 18%] PASSED test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass [gw9] [ 19%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout [gw9] [ 20%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout [gw9] [ 21%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_grpc_protocol_ssl/test.py::test_insecure_channel test_external_cluster/test.py::test_ddl [gw3] [ 22%] PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] [gw0] [ 23%] PASSED test_keeper_zookeeper_converter/test.py::test_acls[False] test_keeper_zookeeper_converter/test.py::test_acls[True] test_hot_reload_storage_policy/test.py::test_hot_reload_policy [gw3] [ 24%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] [gw7] [ 25%] PASSED test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel [gw7] [ 26%] PASSED test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate [gw3] [ 27%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection [gw7] [ 28%] PASSED test_grpc_protocol_ssl/test.py::test_wrong_client_certificate test_keeper_session/test.py::test_session_close_shutdown [gw6] [ 29%] PASSED test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart [gw5] [ 30%] PASSED test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated [gw5] [ 31%] PASSED test_external_cluster/test.py::test_ddl_replicated [gw9] [ 32%] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table [gw0] [ 33%] PASSED test_keeper_zookeeper_converter/test.py::test_acls[True] test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] test_keeper_disks/test.py::test_logs_with_disks [gw4] [ 34%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] [gw4] [ 35%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format [gw6] [ 36%] PASSED test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart [gw4] [ 37%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] [gw4] [ 38%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] [gw3] [ 39%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] [gw4] [ 40%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] [gw2] [ 41%] PASSED test_filesystem_cache/test.py::test_custom_cached_disk test_filesystem_cache/test.py::test_dynamic_resize [gw9] [ 42%] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_keeper_internal_secure/test.py::test_secure_raft_works [gw3] [ 43%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] [gw6] [ 44%] PASSED test_keeper_persistent_log/test.py::test_state_duplicate_restart [gw4] [ 45%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] test_executable_user_defined_functions_config_reload/test.py::test [gw0] [ 46%] PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] [gw2] [ 47%] PASSED test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log [gw3] [ 48%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_ttl [gw7] [ 49%] PASSED test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout [gw7] [ 50%] PASSED test_keeper_session/test.py::test_session_timeout [gw2] [ 51%] PASSED test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_http_and_readonly/test.py::test_http_get_is_readonly [gw5] [ 52%] PASSED test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks [gw0] [ 53%] PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] [gw3] [ 54%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica [gw1] [ 55%] PASSED test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage [gw3] [ 56%] PASSED test_quorum_inserts/test.py::test_simple_add_replica [gw1] [ 57%] PASSED test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 [gw9] [ 58%] PASSED test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password [gw4] [ 59%] PASSED test_http_and_readonly/test.py::test_http_get_is_readonly test_http_native/test.py::test_http_native_returns_timezone [gw0] [ 60%] PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] [gw6] [ 61%] PASSED test_executable_user_defined_functions_config_reload/test.py::test test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts test_jbod_ha/test.py::test_jbod_ha [gw4] [ 62%] PASSED test_http_native/test.py::test_http_native_returns_timezone [gw5] [ 63%] PASSED test_keeper_disks/test.py::test_snapshots_with_disks [gw7] [ 64%] PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_keeper_compression/test_with_compression.py::test_select [gw6] [ 65%] PASSED test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts [gw0] [ 66%] PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] test_keeper_zookeeper_converter/test.py::test_smoke[False] [gw4] [ 67%] PASSED test_keeper_compression/test_with_compression.py::test_select test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_keeper_nodes_move/test.py::test_node_move test_explain_estimates/test.py::test_explain_estimates [gw0] [ 68%] PASSED test_keeper_zookeeper_converter/test.py::test_smoke[False] test_keeper_zookeeper_converter/test.py::test_smoke[True] [gw7] [ 69%] PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_keeper_block_acl/test.py::test_keeper_block_acl [gw1] [ 70%] PASSED test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user [gw5] [ 71%] PASSED test_explain_estimates/test.py::test_explain_estimates [gw1] [ 72%] PASSED test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database test_keeper_four_word_command/test_allow_list.py::test_allow_list [gw3] [ 73%] PASSED test_jbod_ha/test.py::test_jbod_ha [gw6] [ 74%] PASSED test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success [gw1] [ 75%] PASSED test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback [gw0] [ 76%] PASSED test_keeper_zookeeper_converter/test.py::test_smoke[True] [gw1] [ 77%] PASSED test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table [gw5] [ 78%] PASSED test_keeper_four_word_command/test_allow_list.py::test_allow_list [gw8] [ 79%] PASSED test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster [gw1] [ 80%] FAILED test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions test_keeper_memory_soft_limit/test.py::test_soft_limit_create [gw9] [ 81%] PASSED test_keeper_internal_secure/test.py::test_secure_raft_works_with_password [gw1] [ 82%] PASSED test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_inherit_multiple_profiles/test.py::test_combined_profile [gw2] [ 83%] PASSED test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio [gw2] [ 84%] PASSED test_filesystem_cache/test.py::test_keep_up_size_ratio test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] [gw4] [ 85%] PASSED test_keeper_nodes_move/test.py::test_node_move [gw9] [ 86%] PASSED test_inherit_multiple_profiles/test.py::test_combined_profile [gw3] [ 87%] SKIPPED test_keeper_memory_soft_limit/test.py::test_soft_limit_create test_keeper_mntr_pressure/test.py::test_aggressive_mntr [gw6] [ 88%] PASSED test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback [gw1] [ 89%] PASSED test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster [gw1] [ 90%] PASSED test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers [gw7] [ 91%] PASSED test_keeper_block_acl/test.py::test_keeper_block_acl [gw1] [ 92%] PASSED test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions [gw1] [ 93%] FAILED test_mask_sensitive_info/test.py::test_table_functions [gw2] [ 94%] PASSED test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] test_filesystem_cache/test.py::test_system_sync_filesystem_cache [gw2] [ 95%] PASSED test_filesystem_cache/test.py::test_system_sync_filesystem_cache [gw3] [ 96%] PASSED test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers [gw9] [ 97%] PASSED test_keeper_mntr_pressure/test.py::test_aggressive_mntr [gw8] [ 98%] PASSED test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload [gw8] [ 99%] PASSED test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster [gw8] [100%] FAILED test_reload_clusters_config/test.py::test_update_one_cluster =================================== FAILURES =================================== ______________________________ test_create_table _______________________________ [gw1] linux -- Python 3.10.12 /usr/bin/python3 def test_create_table(): password = new_password() azure_conn_string = cluster.env_variables["AZURITE_CONNECTION_STRING"] account_key_pattern = re.compile("AccountKey=.*?(;|$)") masked_azure_conn_string = re.sub( account_key_pattern, "AccountKey=[HIDDEN]\\1", azure_conn_string ) masked_sas_conn_string = 'BlobEndpoint=https://my-endpoint/;SharedAccessSignature=[HIDDEN]' azure_storage_account_url = cluster.env_variables["AZURITE_STORAGE_ACCOUNT_URL"] azure_account_name = "devstoreaccount1" azure_account_key = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" table_engines = [ f"MySQL('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '{password}')", f"PostgreSQL('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '{password}')", f"MongoDB('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '{password}')", f"S3('http://minio1:9001/root/data/test1.csv')", f"S3('http://minio1:9001/root/data/test2.csv', 'CSV')", f"S3('http://minio1:9001/root/data/test3.csv.gz', 'CSV', 'gzip')", f"S3('http://minio1:9001/root/data/test4.csv', 'minio', '{password}', 'CSV')", f"S3('http://minio1:9001/root/data/test5.csv.gz', 'minio', '{password}', 'CSV', 'gzip')", f"MySQL(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '{password}')", f"MySQL(named_collection_2, database = 'mysql_db', host = 'mysql80', port = 3306, password = '{password}', table = 'mysql_table', user = 'mysql_user')", f"MySQL(named_collection_3, database = 'mysql_db', host = 'mysql80', port = 3306, table = 'mysql_table')", f"PostgreSQL(named_collection_4, host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user', password = '{password}')", f"MongoDB(named_collection_5, host = 'mongo1', port = 5432, database = 'mongo_db', collection = 'mongo_col', user = 'mongo_user', password = '{password}')", f"S3(named_collection_6, url = 'http://minio1:9001/root/data/test8.csv', access_key_id = 'minio', secret_access_key = '{password}', format = 'CSV')", f"S3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV', 'gzip')", f"S3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '{password}')", ( f"DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), f"S3Queue('http://minio1:9001/root/data/', 'CSV') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'CSV', 'gzip') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'minio', '{password}', 'CSV') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'minio', '{password}', 'CSV', 'gzip') settings mode = 'ordered'", ( f"Iceberg('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), ( f"IcebergS3('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), f"AzureBlobStorage('{azure_conn_string}', 'cont', 'test_simple.csv', 'CSV')", f"AzureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_1.csv', 'CSV', 'none')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_2.csv', '{azure_account_name}', '{azure_account_key}')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_3.csv', '{azure_account_name}', '{azure_account_key}', 'CSV')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_4.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none')", f"AzureQueue('{azure_conn_string}', 'cont', '*', 'CSV') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_conn_string}', 'cont', '*', 'CSV', 'none') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_storage_account_url}', 'cont', '*', '{azure_account_name}', '{azure_account_key}', 'CSV') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_storage_account_url}', 'cont', '*', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none') SETTINGS mode = 'unordered'", ( f"AzureBlobStorage('BlobEndpoint=https://my-endpoint/;SharedAccessSignature=sp=r&st=2025-09-29T14:58:11Z&se=2025-09-29T00:00:00Z&spr=https&sv=2022-11-02&sr=c&sig=SECRET%SECRET%SECRET%SECRET', 'exampledatasets', 'example.csv')", "STD_EXCEPTION", ), f"Kafka() SETTINGS kafka_broker_list = '127.0.0.1', kafka_topic_list = 'topic', kafka_group_name = 'group', kafka_format = 'JSONEachRow', format_avro_schema_registry_url = 'http://schema_user:{password}@'", f"Kafka() SETTINGS kafka_broker_list = '127.0.0.1', kafka_topic_list = 'topic', kafka_group_name = 'group', kafka_format = 'JSONEachRow', format_avro_schema_registry_url = 'http://schema_user:{password}@domain.com'", ] def make_test_case(i): table_name = f"table{i}" table_engine = table_engines[i] error = None if isinstance(table_engine, tuple): table_engine, error = table_engine query = f"CREATE TABLE {table_name} (x int) ENGINE = {table_engine}" return table_name, query, error # Generate test cases as a list of tuples (table_name, query, error). test_cases = [make_test_case(i) for i in range(len(table_engines))] for table_name, query, error in test_cases: if error: > assert error in node.query_and_get_error(query) E assert 'DNS_ERROR' in "Received exception from server (version 25.3.8):\nCode: 56. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table engine DeltaLake. Stack trace:\n\n0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x000000001f81eb40\n1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000ff71534\n2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007c24d20\n3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x0000000007c44280\n4. ./build_docker/./src/Storages/StorageFactory.cpp:173: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel, bool) const @ 0x0000000019ee4265\n5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1945: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x0000000017cf8454\n6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1743: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000017cef9b9\n7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:2355: DB::InterpreterCreateQuery::execute() @ 0x0000000017cff63b\n8. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000001824006e\n9. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001823ac6f\n10. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000001b014245\n11. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000001b03afa8\n12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000001f92c623\n13. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000001f92ce92\n14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000001f8a7143\n15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:45: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001f8a54f0\n16. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001f8a38aa\n17. __tsan_thread_start_func @ 0x0000000007b98428\n18. ? @ 0x00007fd53a9c1ac3\n19. ? @ 0x00007fd53aa53850\n. (UNKNOWN_STORAGE)\n(query: CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'QDKOZI56H17KRO46'))\n" E + where "Received exception from server (version 25.3.8):\nCode: 56. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table engine DeltaLake. Stack trace:\n\n0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x000000001f81eb40\n1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000ff71534\n2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007c24d20\n3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x0000000007c44280\n4. ./build_docker/./src/Storages/StorageFactory.cpp:173: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel, bool) const @ 0x0000000019ee4265\n5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1945: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x0000000017cf8454\n6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1743: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000017cef9b9\n7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:2355: DB::InterpreterCreateQuery::execute() @ 0x0000000017cff63b\n8. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000001824006e\n9. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001823ac6f\n10. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000001b014245\n11. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000001b03afa8\n12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000001f92c623\n13. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000001f92ce92\n14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000001f8a7143\n15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:45: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001f8a54f0\n16. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001f8a38aa\n17. __tsan_thread_start_func @ 0x0000000007b98428\n18. ? @ 0x00007fd53a9c1ac3\n19. ? @ 0x00007fd53aa53850\n. (UNKNOWN_STORAGE)\n(query: CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'QDKOZI56H17KRO46'))\n" = >("CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'QDKOZI56H17KRO46')") E + where > = .query_and_get_error test_mask_sensitive_info/test.py:309: AssertionError ------------------------------ Captured log call ------------------------------- 2026-04-10 18:33:27.512000 [ 674 ] DEBUG : Executing query CREATE TABLE table0 (x int) ENGINE = MySQL('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:27.727000 [ 674 ] DEBUG : Executing query CREATE TABLE table1 (x int) ENGINE = PostgreSQL('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:28.243000 [ 674 ] DEBUG : Executing query CREATE TABLE table2 (x int) ENGINE = MongoDB('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:28.458000 [ 674 ] DEBUG : Executing query CREATE TABLE table3 (x int) ENGINE = S3('http://minio1:9001/root/data/test1.csv') on node (cluster.py:3658, query) 2026-04-10 18:33:28.775000 [ 674 ] DEBUG : Executing query CREATE TABLE table4 (x int) ENGINE = S3('http://minio1:9001/root/data/test2.csv', 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:33:29.040000 [ 674 ] DEBUG : Executing query CREATE TABLE table5 (x int) ENGINE = S3('http://minio1:9001/root/data/test3.csv.gz', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 18:33:29.306000 [ 674 ] DEBUG : Executing query CREATE TABLE table6 (x int) ENGINE = S3('http://minio1:9001/root/data/test4.csv', 'minio', 'QDKOZI56H17KRO46', 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:33:29.672000 [ 674 ] DEBUG : Executing query CREATE TABLE table7 (x int) ENGINE = S3('http://minio1:9001/root/data/test5.csv.gz', 'minio', 'QDKOZI56H17KRO46', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 18:33:29.938000 [ 674 ] DEBUG : Executing query CREATE TABLE table8 (x int) ENGINE = MySQL(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:30.153000 [ 674 ] DEBUG : Executing query CREATE TABLE table9 (x int) ENGINE = MySQL(named_collection_2, database = 'mysql_db', host = 'mysql80', port = 3306, password = 'QDKOZI56H17KRO46', table = 'mysql_table', user = 'mysql_user') on node (cluster.py:3658, query) 2026-04-10 18:33:30.369000 [ 674 ] DEBUG : Executing query CREATE TABLE table10 (x int) ENGINE = MySQL(named_collection_3, database = 'mysql_db', host = 'mysql80', port = 3306, table = 'mysql_table') on node (cluster.py:3658, query) 2026-04-10 18:33:30.635000 [ 674 ] DEBUG : Executing query CREATE TABLE table11 (x int) ENGINE = PostgreSQL(named_collection_4, host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user', password = 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:30.850000 [ 674 ] DEBUG : Executing query CREATE TABLE table12 (x int) ENGINE = MongoDB(named_collection_5, host = 'mongo1', port = 5432, database = 'mongo_db', collection = 'mongo_col', user = 'mongo_user', password = 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:31.066000 [ 674 ] DEBUG : Executing query CREATE TABLE table13 (x int) ENGINE = S3(named_collection_6, url = 'http://minio1:9001/root/data/test8.csv', access_key_id = 'minio', secret_access_key = 'QDKOZI56H17KRO46', format = 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:33:31.331000 [ 674 ] DEBUG : Executing query CREATE TABLE table14 (x int) ENGINE = S3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 18:33:31.697000 [ 674 ] DEBUG : Executing query CREATE TABLE table15 (x int) ENGINE = S3('http://minio1:9001/root/data/test10.csv.gz', 'minio', 'QDKOZI56H17KRO46') on node (cluster.py:3658, query) 2026-04-10 18:33:31.963000 [ 674 ] DEBUG : Executing query CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'QDKOZI56H17KRO46') on node (cluster.py:3741, query_and_get_error) _____________________________ test_table_functions _____________________________ [gw1] linux -- Python 3.10.12 /usr/bin/python3 def test_table_functions(): password = new_password() azure_conn_string = cluster.env_variables["AZURITE_CONNECTION_STRING"] account_key_pattern = re.compile("AccountKey=.*?(;|$)") masked_azure_conn_string = re.sub( account_key_pattern, "AccountKey=[HIDDEN]\\1", azure_conn_string ) azure_storage_account_url = cluster.env_variables["AZURITE_STORAGE_ACCOUNT_URL"] azure_account_name = "devstoreaccount1" azure_account_key = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" table_functions = [ f"mysql('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '{password}')", f"postgresql('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '{password}')", f"mongodb('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '{password}', 'x int')", f"s3('http://minio1:9001/root/data/test1.csv')", f"s3('http://minio1:9001/root/data/test2.csv', 'CSV')", f"s3('http://minio1:9001/root/data/test3.csv', 'minio', '{password}')", f"s3('http://minio1:9001/root/data/test4.csv', 'CSV', 'x int')", f"s3('http://minio1:9001/root/data/test5.csv.gz', 'CSV', 'x int', 'gzip')", f"s3('http://minio1:9001/root/data/test6.csv', 'minio', '{password}', 'CSV')", f"s3('http://minio1:9001/root/data/test7.csv', 'minio', '{password}', 'CSV', 'x int')", f"s3('http://minio1:9001/root/data/test8.csv.gz', 'minio', '{password}', 'CSV', 'x int', 'gzip')", f"remote('127.{{2..11}}', default.remote_table)", f"remote('127.{{2..11}}', default.remote_table, rand())", f"remote('127.{{2..11}}', default.remote_table, 'remote_user')", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', '{password}')", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', rand())", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', 'default.remote_table', 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', 'default', 'remote_table', 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', numbers(10), 'remote_user', '{password}', rand())", f"remoteSecure('127.{{2..11}}', 'default', 'remote_table', 'remote_user', '{password}')", f"remoteSecure('127.{{2..11}}', 'default', 'remote_table', 'remote_user', rand())", f"mysql(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '{password}')", f"postgresql(named_collection_2, password = '{password}', host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user')", f"s3(named_collection_2, url = 'http://minio1:9001/root/data/test4.csv', access_key_id = 'minio', secret_access_key = '{password}')", f"remote(named_collection_6, addresses_expr = '127.{{2..11}}', database = 'default', table = 'remote_table', user = 'remote_user', password = '{password}', sharding_key = rand())", f"remoteSecure(named_collection_6, addresses_expr = '127.{{2..11}}', database = 'default', table = 'remote_table', user = 'remote_user', password = '{password}')", f"s3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV')", f"s3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '{password}')", f"deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple.csv', 'CSV')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_1.csv', 'CSV', 'none')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_2.csv', 'CSV', 'none', 'auto')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_3.csv', '{azure_account_name}', '{azure_account_key}')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_4.csv', '{azure_account_name}', '{azure_account_key}', 'CSV')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_5.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_6.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none', 'auto')", f"azureBlobStorage(named_collection_2, connection_string = '{azure_conn_string}', container = 'cont', blob_path = 'test_simple_7.csv', format = 'CSV')", f"azureBlobStorage(named_collection_2, storage_account_url = '{azure_storage_account_url}', container = 'cont', blob_path = 'test_simple_8.csv', account_name = '{azure_account_name}', account_key = '{azure_account_key}')", f"iceberg('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"gcs('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"icebergS3('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"icebergAzure('{azure_storage_account_url}', 'cont', 'test_simple_6.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none', 'auto')", ] def make_test_case(i): table_name = f"tablefunc{i}" table_function = table_functions[i] error = None if isinstance(table_function, tuple): table_function, error = table_function query = f"CREATE TABLE {table_name} (x int) AS {table_function}" return table_name, query, error # Generate test cases as a list of tuples (table_name, query, error). test_cases = [make_test_case(i) for i in range(len(table_functions))] for table_name, query, error in test_cases: > node.query(query) test_mask_sensitive_info/test.py:503: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 46, stderr: Received exception from server (version 25.3.8): E Code: 46. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table function deltaLake. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x000000001f81eb40 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000ff71534 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007c24d20 E 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000894ef40 E 4. ./build_docker/./src/TableFunctions/TableFunctionFactory.cpp:50: DB::TableFunctionFactory::get(std::shared_ptr const&, std::shared_ptr) const @ 0x00000000158c6ca1 E 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1932: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x0000000017cf8079 E 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1743: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000017cef9b9 E 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:2355: DB::InterpreterCreateQuery::execute() @ 0x0000000017cff63b E 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000001824006e E 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001823ac6f E 10. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000001b014245 E 11. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000001b03afa8 E 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000001f92c623 E 13. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000001f92ce92 E 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000001f8a7143 E 15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:45: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001f8a54f0 E 16. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001f8a38aa E 17. __tsan_thread_start_func @ 0x0000000007b98428 E 18. ? @ 0x00007fd53a9c1ac3 E 19. ? @ 0x00007fd53aa53850 E . (UNKNOWN_FUNCTION) E (query: CREATE TABLE tablefunc29 (x int) AS deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '7VZ9L89IGNZ8W554')) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 18:34:23.982000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc0 (x int) AS mysql('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:24.302000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc1 (x int) AS postgresql('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:24.619000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc2 (x int) AS mongodb('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '7VZ9L89IGNZ8W554', 'x int') on node (cluster.py:3658, query) 2026-04-10 18:34:24.985000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc3 (x int) AS s3('http://minio1:9001/root/data/test1.csv') on node (cluster.py:3658, query) 2026-04-10 18:34:25.253000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc4 (x int) AS s3('http://minio1:9001/root/data/test2.csv', 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:34:25.519000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc5 (x int) AS s3('http://minio1:9001/root/data/test3.csv', 'minio', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:25.835000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc6 (x int) AS s3('http://minio1:9001/root/data/test4.csv', 'CSV', 'x int') on node (cluster.py:3658, query) 2026-04-10 18:34:26.155000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc7 (x int) AS s3('http://minio1:9001/root/data/test5.csv.gz', 'CSV', 'x int', 'gzip') on node (cluster.py:3658, query) 2026-04-10 18:34:26.425000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc8 (x int) AS s3('http://minio1:9001/root/data/test6.csv', 'minio', '7VZ9L89IGNZ8W554', 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:34:26.741000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc9 (x int) AS s3('http://minio1:9001/root/data/test7.csv', 'minio', '7VZ9L89IGNZ8W554', 'CSV', 'x int') on node (cluster.py:3658, query) 2026-04-10 18:34:27.058000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc10 (x int) AS s3('http://minio1:9001/root/data/test8.csv.gz', 'minio', '7VZ9L89IGNZ8W554', 'CSV', 'x int', 'gzip') on node (cluster.py:3658, query) 2026-04-10 18:34:27.426000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc11 (x int) AS remote('127.{2..11}', default.remote_table) on node (cluster.py:3658, query) 2026-04-10 18:34:27.742000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc12 (x int) AS remote('127.{2..11}', default.remote_table, rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:28.009000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc13 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user') on node (cluster.py:3658, query) 2026-04-10 18:34:28.377000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc14 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:28.644000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc15 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:28.910000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc16 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', '7VZ9L89IGNZ8W554', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:29.175000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc17 (x int) AS remote('127.{2..11}', 'default.remote_table', 'remote_user', '7VZ9L89IGNZ8W554', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:29.491000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc18 (x int) AS remote('127.{2..11}', 'default', 'remote_table', 'remote_user', '7VZ9L89IGNZ8W554', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:29.707000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc19 (x int) AS remote('127.{2..11}', numbers(10), 'remote_user', '7VZ9L89IGNZ8W554', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:30.022000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc20 (x int) AS remoteSecure('127.{2..11}', 'default', 'remote_table', 'remote_user', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:30.237000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc21 (x int) AS remoteSecure('127.{2..11}', 'default', 'remote_table', 'remote_user', rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:30.453000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc22 (x int) AS mysql(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:30.719000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc23 (x int) AS postgresql(named_collection_2, password = '7VZ9L89IGNZ8W554', host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user') on node (cluster.py:3658, query) 2026-04-10 18:34:30.934000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc24 (x int) AS s3(named_collection_2, url = 'http://minio1:9001/root/data/test4.csv', access_key_id = 'minio', secret_access_key = '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:31.200000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc25 (x int) AS remote(named_collection_6, addresses_expr = '127.{2..11}', database = 'default', table = 'remote_table', user = 'remote_user', password = '7VZ9L89IGNZ8W554', sharding_key = rand()) on node (cluster.py:3658, query) 2026-04-10 18:34:31.415000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc26 (x int) AS remoteSecure(named_collection_6, addresses_expr = '127.{2..11}', database = 'default', table = 'remote_table', user = 'remote_user', password = '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:31.681000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc27 (x int) AS s3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV') on node (cluster.py:3658, query) 2026-04-10 18:34:31.947000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc28 (x int) AS s3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) 2026-04-10 18:34:32.163000 [ 674 ] DEBUG : Executing query CREATE TABLE tablefunc29 (x int) AS deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '7VZ9L89IGNZ8W554') on node (cluster.py:3658, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:34:32.638000 [ 674 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/.env --project-name roottestmasksensitiveinfo-gw1 --file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:48.915000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:48.916000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:48.916000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:48.916000 [ 674 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:34:48.930000 [ 674 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/.env --project-name roottestmasksensitiveinfo-gw1 --file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw1/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:34:49.528000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:49.528000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:49.528000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-node-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.529000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw1-azurite1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Volume roottestmasksensitiveinfo-gw1_data1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Network roottestmasksensitiveinfo-gw1_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Volume roottestmasksensitiveinfo-gw1_data1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Stderr: Network roottestmasksensitiveinfo-gw1_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:34:49.530000 [ 674 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:34:49.554000 [ 674 ] DEBUG : Docker networks for project roottestmasksensitiveinfo-gw1 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:34:49.574000 [ 674 ] DEBUG : Docker containers for project roottestmasksensitiveinfo-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:34:49.595000 [ 674 ] DEBUG : Docker volumes for project roottestmasksensitiveinfo-gw1 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:34:49.595000 [ 674 ] DEBUG : Command:[docker container list --all --filter name='^/roottestmasksensitiveinfo-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:34:49.615000 [ 674 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:34:49.615000 [ 674 ] DEBUG : No running containers for project: roottestmasksensitiveinfo-gw1 (cluster.py:889, cleanup) 2026-04-10 18:34:49.615000 [ 674 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:34:49.635000 [ 674 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:34:49.635000 [ 674 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:34:49.661000 [ 674 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:34:49.661000 [ 674 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:34:49.661000 [ 674 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:34:49.661000 [ 674 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:34:49.679000 [ 674 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:34:49.680000 [ 674 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) ___________________________ test_update_one_cluster ____________________________ [gw8] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_update_one_cluster(started_cluster): send_repeated_query("distributed") send_repeated_query("distributed2") > assert get_errors_count("test_cluster") > 0 E AssertionError: assert 0 > 0 E + where 0 = get_errors_count('test_cluster') test_reload_clusters_config/test.py:202: AssertionError ------------------------------ Captured log call ------------------------------- 2026-04-10 18:36:50.950000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:37:03.206000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:37:16.217000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:37:28.529000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:37:40.940000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:37:54.951000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:38:07.960000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:38:20.215000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:38:32.520000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:38:45.476000 [ 725 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 18:38:57.730000 [ 725 ] DEBUG : Executing query SELECT errors_count FROM system.clusters WHERE cluster='test_cluster' and host_name='node_1' on node (cluster.py:3658, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:38:58.070000 [ 725 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/.env --project-name roottestreloadclustersconfig-gw8 --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.404000 [ 725 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:39:05.414000 [ 725 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/.env --project-name roottestreloadclustersconfig-gw8 --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:39:05.891000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:39:05.892000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Stderr: Network roottestreloadclustersconfig-gw8_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Stderr: Network roottestreloadclustersconfig-gw8_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:39:05.893000 [ 725 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:39:05.916000 [ 725 ] DEBUG : Docker networks for project roottestreloadclustersconfig-gw8 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:39:05.938000 [ 725 ] DEBUG : Docker containers for project roottestreloadclustersconfig-gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:39:05.956000 [ 725 ] DEBUG : Docker volumes for project roottestreloadclustersconfig-gw8 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:39:05.956000 [ 725 ] DEBUG : Command:[docker container list --all --filter name='^/roottestreloadclustersconfig-gw8-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:39:05.977000 [ 725 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:39:05.978000 [ 725 ] DEBUG : No running containers for project: roottestreloadclustersconfig-gw8 (cluster.py:889, cleanup) 2026-04-10 18:39:05.978000 [ 725 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:39:06.000000 [ 725 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:39:06.000000 [ 725 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:39:06.030000 [ 725 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:39:06.031000 [ 725 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:39:06.031000 [ 725 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:39:06.031000 [ 725 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:39:06.053000 [ 725 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:39:06.053000 [ 725 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) ----------------- generated report log file: parallel1_0.jsonl ----------------- ============================== slowest durations =============================== 142.24s call test_reload_clusters_config/test.py::test_add_cluster 136.52s call test_reload_clusters_config/test.py::test_delete_cluster 127.05s call test_reload_clusters_config/test.py::test_update_one_cluster 109.93s call test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 91.75s setup test_mask_sensitive_info/test.py::test_backup_table 82.71s call test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 67.73s call test_keeper_mntr_pressure/test.py::test_aggressive_mntr 63.08s call test_reload_clusters_config/test.py::test_simple_reload 55.65s call test_mask_sensitive_info/test.py::test_backup_to_s3 45.90s call test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 45.59s call test_keeper_block_acl/test.py::test_keeper_block_acl 41.86s call test_jbod_ha/test.py::test_jbod_ha 34.93s call test_keeper_nodes_move/test.py::test_node_move 31.47s setup test_reload_clusters_config/test.py::test_add_cluster 30.17s call test_keeper_disks/test.py::test_snapshots_with_disks 29.89s call test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 28.40s call test_keeper_disks/test.py::test_logs_with_disks 27.54s call test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 25.26s call test_filesystem_cache/test.py::test_custom_cached_disk 24.88s setup test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 24.65s call test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 23.63s setup test_keeper_block_acl/test.py::test_keeper_block_acl 23.47s call test_keeper_zookeeper_converter/test.py::test_smoke[True] 23.13s setup test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 23.11s call test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 22.76s call test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 22.05s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 22.02s call test_executable_user_defined_functions_config_reload/test.py::test 21.85s teardown test_keeper_disks/test.py::test_snapshots_with_disks 20.23s setup test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 20.11s call test_keeper_internal_secure/test.py::test_secure_raft_works 19.95s call test_mask_sensitive_info/test.py::test_table_function_ways_to_call 19.93s call test_quorum_inserts/test.py::test_insert_quorum_with_ttl 19.81s call test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 19.68s setup test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 19.18s call test_mask_sensitive_info/test.py::test_iceberg_cluster_function 19.11s call test_keeper_zookeeper_converter/test.py::test_acls[True] 19.01s teardown test_quorum_inserts/test.py::test_simple_add_replica 18.93s setup test_executable_user_defined_functions_config_reload/test.py::test 18.73s setup test_keeper_memory_soft_limit/test.py::test_soft_limit_create 18.33s setup test_keeper_nodes_move/test.py::test_node_move 17.65s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 17.48s teardown test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 17.42s call test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 17.35s teardown test_keeper_memory_soft_limit/test.py::test_soft_limit_create 17.04s teardown test_mask_sensitive_info/test.py::test_table_functions 16.83s setup test_keeper_mntr_pressure/test.py::test_aggressive_mntr 16.70s setup test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 16.65s call test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 16.62s setup test_keeper_session/test.py::test_session_close_shutdown 16.22s setup test_keeper_persistent_log/test.py::test_ephemeral_after_restart 16.14s setup test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 16.04s setup test_keeper_internal_secure/test.py::test_secure_raft_works 15.97s setup test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 15.88s setup test_keeper_compression/test_with_compression.py::test_select 15.85s setup test_keeper_zookeeper_converter/test.py::test_acls[False] 15.78s setup test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 15.63s teardown test_jbod_ha/test.py::test_jbod_ha 15.40s call test_keeper_session/test.py::test_session_close_shutdown 15.13s call test_keeper_zookeeper_converter/test.py::test_smoke[False] 14.51s setup test_explain_estimates/test.py::test_explain_estimates 14.45s setup test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 14.36s setup test_http_native/test.py::test_http_native_returns_timezone 14.29s call test_keeper_persistent_log/test.py::test_state_duplicate_restart 13.95s setup test_external_http_authenticator/test.py::test_basic_auth_failed 13.84s setup test_hot_reload_storage_policy/test.py::test_hot_reload_policy 13.78s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 13.75s setup test_http_and_readonly/test.py::test_http_get_is_readonly 13.73s call test_keeper_zookeeper_converter/test.py::test_acls[False] 13.68s call test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 12.99s call test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 12.16s teardown test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 11.80s teardown test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 11.71s setup test_filesystem_cache/test.py::test_custom_cached_disk 11.53s setup test_inherit_multiple_profiles/test.py::test_combined_profile 10.99s setup test_keeper_disks/test.py::test_logs_with_disks 10.96s call test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 10.70s call test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 10.46s setup test_external_cluster/test.py::test_ddl 9.61s setup test_jbod_ha/test.py::test_jbod_ha 8.93s call test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 8.87s call test_filesystem_cache/test.py::test_dynamic_resize 8.56s call test_mask_sensitive_info/test.py::test_table_functions 8.39s call test_mask_sensitive_info/test.py::test_encryption_functions 8.20s call test_filesystem_cache/test.py::test_system_sync_filesystem_cache 7.98s teardown test_reload_clusters_config/test.py::test_update_one_cluster 7.95s call test_mask_sensitive_info/test.py::test_create_alter_user 7.65s setup test_keeper_four_word_command/test_allow_list.py::test_allow_list 7.62s call test_external_http_authenticator/test.py::test_session_settings_from_auth_response 7.48s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 7.24s call test_keeper_persistent_log/test.py::test_ephemeral_after_restart 6.65s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 6.36s call test_keeper_persistent_log/test.py::test_state_after_restart 6.28s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 6.27s teardown test_filesystem_cache/test.py::test_system_sync_filesystem_cache 6.23s setup test_grpc_protocol_ssl/test.py::test_insecure_channel 5.92s call test_filesystem_cache/test.py::test_keep_up_size_ratio 5.54s call test_mask_sensitive_info/test.py::test_create_database 5.17s call test_quorum_inserts/test.py::test_simple_add_replica 5.12s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 5.10s teardown test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 4.77s call test_mask_sensitive_info/test.py::test_create_table 4.60s teardown test_external_cluster/test.py::test_ddl_replicated 4.58s call test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 4.52s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 4.49s call test_mask_sensitive_info/test.py::test_create_dictionary 4.37s call test_hot_reload_storage_policy/test.py::test_hot_reload_policy 4.29s call test_filesystem_cache/test.py::test_filesystem_cache_log 4.02s call test_mask_sensitive_info/test.py::test_on_cluster 3.92s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 3.89s call test_external_cluster/test.py::test_ddl 3.89s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 3.58s teardown test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 3.57s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 3.49s teardown test_explain_estimates/test.py::test_explain_estimates 3.48s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync 3.32s call test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 3.23s teardown test_inherit_multiple_profiles/test.py::test_combined_profile 3.14s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 2.93s teardown test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 2.92s call test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 2.77s teardown test_http_native/test.py::test_http_native_returns_timezone 2.71s call test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 2.70s call test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 2.57s teardown test_http_and_readonly/test.py::test_http_get_is_readonly 2.56s call test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 2.43s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 2.43s teardown test_keeper_block_acl/test.py::test_keeper_block_acl 2.19s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 2.15s call test_inherit_multiple_profiles/test.py::test_combined_profile 2.02s call test_grpc_protocol_ssl/test.py::test_insecure_channel 2.00s call test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 2.00s teardown test_keeper_nodes_move/test.py::test_node_move 1.85s call test_explain_estimates/test.py::test_explain_estimates 1.81s teardown test_keeper_mntr_pressure/test.py::test_aggressive_mntr 1.79s call test_mask_sensitive_info/test.py::test_backup_table 1.79s teardown test_keeper_zookeeper_converter/test.py::test_smoke[True] 1.77s teardown test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 1.71s teardown test_keeper_session/test.py::test_session_timeout 1.68s call test_keeper_four_word_command/test_allow_list.py::test_allow_list 1.66s teardown test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 1.66s teardown test_keeper_four_word_command/test_allow_list.py::test_allow_list 1.64s teardown test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 1.58s teardown test_keeper_compression/test_with_compression.py::test_select 1.57s call test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 1.43s teardown test_filesystem_cache/test.py::test_custom_cached_disk 1.40s call test_external_cluster/test.py::test_ddl_replicated 1.35s teardown test_executable_user_defined_functions_config_reload/test.py::test 1.34s teardown test_keeper_persistent_log/test.py::test_state_duplicate_restart 1.27s call test_external_http_authenticator/test.py::test_user_create_basic_auth_pass 1.26s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 1.02s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.95s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 0.68s call test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 0.51s call test_http_and_readonly/test.py::test_http_get_is_readonly 0.47s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.47s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 0.47s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.42s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.42s call test_keeper_memory_soft_limit/test.py::test_soft_limit_create 0.38s call test_http_native/test.py::test_http_native_returns_timezone 0.38s call test_keeper_session/test.py::test_session_timeout 0.37s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.32s call test_external_http_authenticator/test.py::test_basic_auth_failed 0.32s call test_keeper_compression/test_with_compression.py::test_select 0.24s call test_grpc_protocol_ssl/test.py::test_secure_channel 0.06s call test_external_http_authenticator/test.py::test_header_failed 0.00s teardown test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 0.00s teardown test_keeper_persistent_log/test.py::test_ephemeral_after_restart 0.00s teardown test_keeper_zookeeper_converter/test.py::test_acls[True] 0.00s teardown test_hot_reload_storage_policy/test.py::test_hot_reload_policy 0.00s teardown test_keeper_persistent_log/test.py::test_state_after_restart 0.00s setup test_keeper_persistent_log/test.py::test_state_duplicate_restart 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_smoke[False] 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 0.00s teardown test_keeper_internal_secure/test.py::test_secure_raft_works 0.00s teardown test_keeper_disks/test.py::test_logs_with_disks 0.00s setup test_keeper_persistent_log/test.py::test_state_after_restart 0.00s setup test_keeper_disks/test.py::test_snapshots_with_disks 0.00s setup test_quorum_inserts/test.py::test_simple_add_replica 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_mask_sensitive_info/test.py::test_backup_to_s3 0.00s teardown test_keeper_zookeeper_converter/test.py::test_acls[False] 0.00s setup test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 0.00s setup test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 0.00s teardown test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.00s setup test_keeper_zookeeper_converter/test.py::test_smoke[True] 0.00s teardown test_external_http_authenticator/test.py::test_basic_auth_failed 0.00s teardown test_mask_sensitive_info/test.py::test_backup_table 0.00s teardown test_reload_clusters_config/test.py::test_add_cluster 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 0.00s setup test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 0.00s teardown test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s setup test_grpc_protocol_ssl/test.py::test_secure_channel 0.00s setup test_filesystem_cache/test.py::test_dynamic_resize 0.00s setup test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 0.00s setup test_external_http_authenticator/test.py::test_header_failed 0.00s setup test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 0.00s teardown test_keeper_session/test.py::test_session_close_shutdown 0.00s setup test_mask_sensitive_info/test.py::test_create_alter_user 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s setup test_mask_sensitive_info/test.py::test_create_database 0.00s setup test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 0.00s setup test_keeper_zookeeper_converter/test.py::test_smoke[False] 0.00s setup test_mask_sensitive_info/test.py::test_table_function_ways_to_call 0.00s setup test_reload_clusters_config/test.py::test_delete_cluster 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s setup test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 0.00s setup test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.00s setup test_keeper_zookeeper_converter/test.py::test_acls[True] 0.00s teardown test_filesystem_cache/test.py::test_keep_up_size_ratio 0.00s setup test_mask_sensitive_info/test.py::test_table_functions 0.00s teardown test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 0.00s teardown test_external_cluster/test.py::test_ddl 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s setup test_keeper_session/test.py::test_session_timeout 0.00s setup test_filesystem_cache/test.py::test_keep_up_size_ratio 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 0.00s teardown test_mask_sensitive_info/test.py::test_encryption_functions 0.00s teardown test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 0.00s setup test_external_http_authenticator/test.py::test_user_create_basic_auth_pass 0.00s teardown test_mask_sensitive_info/test.py::test_table_function_ways_to_call 0.00s setup test_filesystem_cache/test.py::test_system_sync_filesystem_cache 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s setup test_mask_sensitive_info/test.py::test_iceberg_cluster_function 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 0.00s teardown test_mask_sensitive_info/test.py::test_create_table 0.00s setup test_external_cluster/test.py::test_ddl_replicated 0.00s teardown test_mask_sensitive_info/test.py::test_create_alter_user 0.00s setup test_filesystem_cache/test.py::test_filesystem_cache_log 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 0.00s setup test_mask_sensitive_info/test.py::test_on_cluster 0.00s setup test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s setup test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.00s teardown test_filesystem_cache/test.py::test_dynamic_resize 0.00s setup test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync 0.00s teardown test_external_http_authenticator/test.py::test_session_settings_from_auth_response 0.00s setup test_mask_sensitive_info/test.py::test_encryption_functions 0.00s setup test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.00s setup test_mask_sensitive_info/test.py::test_create_table 0.00s setup test_mask_sensitive_info/test.py::test_backup_to_s3 0.00s setup test_reload_clusters_config/test.py::test_update_one_cluster 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync 0.00s setup test_reload_clusters_config/test.py::test_simple_reload 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 0.00s setup test_mask_sensitive_info/test.py::test_create_dictionary 0.00s setup test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 0.00s teardown test_mask_sensitive_info/test.py::test_on_cluster 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.00s teardown test_mask_sensitive_info/test.py::test_create_database 0.00s teardown test_grpc_protocol_ssl/test.py::test_insecure_channel 0.00s call test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_reload_clusters_config/test.py::test_delete_cluster 0.00s setup test_external_http_authenticator/test.py::test_session_settings_from_auth_response 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.00s teardown test_mask_sensitive_info/test.py::test_iceberg_cluster_function 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 0.00s teardown test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 0.00s teardown test_grpc_protocol_ssl/test.py::test_secure_channel 0.00s teardown test_mask_sensitive_info/test.py::test_create_dictionary 0.00s teardown test_reload_clusters_config/test.py::test_simple_reload 0.00s teardown test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s teardown test_filesystem_cache/test.py::test_filesystem_cache_log 0.00s teardown test_external_http_authenticator/test.py::test_user_create_basic_auth_pass 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 0.00s teardown test_external_http_authenticator/test.py::test_header_failed =========================== short test summary info ============================ FAILED test_mask_sensitive_info/test.py::test_create_table - assert 'DNS_ERRO... FAILED test_mask_sensitive_info/test.py::test_table_functions - helpers.clien... FAILED test_reload_clusters_config/test.py::test_update_one_cluster - Asserti... PASSED test_external_http_authenticator/test.py::test_basic_auth_failed PASSED test_external_http_authenticator/test.py::test_header_failed PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] PASSED test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk PASSED test_external_http_authenticator/test.py::test_session_settings_from_auth_response PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin PASSED test_external_http_authenticator/test.py::test_user_create_basic_auth_pass PASSED test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum PASSED test_keeper_zookeeper_converter/test.py::test_acls[False] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] PASSED test_grpc_protocol_ssl/test.py::test_insecure_channel PASSED test_grpc_protocol_ssl/test.py::test_secure_channel PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] PASSED test_grpc_protocol_ssl/test.py::test_wrong_client_certificate PASSED test_keeper_persistent_log/test.py::test_ephemeral_after_restart PASSED test_external_cluster/test.py::test_ddl PASSED test_external_cluster/test.py::test_ddl_replicated PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy PASSED test_keeper_zookeeper_converter/test.py::test_acls[True] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] PASSED test_keeper_persistent_log/test.py::test_state_after_restart PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] PASSED test_filesystem_cache/test.py::test_custom_cached_disk PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] PASSED test_keeper_persistent_log/test.py::test_state_duplicate_restart PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] PASSED test_filesystem_cache/test.py::test_dynamic_resize PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] PASSED test_keeper_session/test.py::test_session_close_shutdown PASSED test_keeper_session/test.py::test_session_timeout PASSED test_filesystem_cache/test.py::test_filesystem_cache_log PASSED test_keeper_disks/test.py::test_logs_with_disks PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl PASSED test_mask_sensitive_info/test.py::test_backup_table PASSED test_quorum_inserts/test.py::test_simple_add_replica PASSED test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage PASSED test_keeper_internal_secure/test.py::test_secure_raft_works PASSED test_http_and_readonly/test.py::test_http_get_is_readonly PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] PASSED test_executable_user_defined_functions_config_reload/test.py::test PASSED test_http_native/test.py::test_http_native_returns_timezone PASSED test_keeper_disks/test.py::test_snapshots_with_disks PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task PASSED test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] PASSED test_keeper_compression/test_with_compression.py::test_select PASSED test_keeper_zookeeper_converter/test.py::test_smoke[False] PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table PASSED test_mask_sensitive_info/test.py::test_backup_to_s3 PASSED test_explain_estimates/test.py::test_explain_estimates PASSED test_mask_sensitive_info/test.py::test_create_alter_user PASSED test_jbod_ha/test.py::test_jbod_ha PASSED test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success PASSED test_mask_sensitive_info/test.py::test_create_database PASSED test_keeper_zookeeper_converter/test.py::test_smoke[True] PASSED test_mask_sensitive_info/test.py::test_create_dictionary PASSED test_keeper_four_word_command/test_allow_list.py::test_allow_list PASSED test_reload_clusters_config/test.py::test_add_cluster PASSED test_keeper_internal_secure/test.py::test_secure_raft_works_with_password PASSED test_mask_sensitive_info/test.py::test_encryption_functions PASSED test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges PASSED test_filesystem_cache/test.py::test_keep_up_size_ratio PASSED test_keeper_nodes_move/test.py::test_node_move PASSED test_inherit_multiple_profiles/test.py::test_combined_profile PASSED test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback PASSED test_mask_sensitive_info/test.py::test_iceberg_cluster_function PASSED test_mask_sensitive_info/test.py::test_on_cluster PASSED test_keeper_block_acl/test.py::test_keeper_block_acl PASSED test_mask_sensitive_info/test.py::test_table_function_ways_to_call PASSED test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] PASSED test_filesystem_cache/test.py::test_system_sync_filesystem_cache PASSED test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers PASSED test_keeper_mntr_pressure/test.py::test_aggressive_mntr PASSED test_reload_clusters_config/test.py::test_delete_cluster PASSED test_reload_clusters_config/test.py::test_simple_reload SKIPPED [1] test_keeper_memory_soft_limit/test.py:46: Disabled for sanitizers ============= 3 failed, 96 passed, 1 skipped in 511.00s (0:08:30) ============== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_xo6f5c --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2]' test_executable_user_defined_functions_config_reload/test.py::test test_explain_estimates/test.py::test_explain_estimates test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node]' 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path]' test_filesystem_cache/test.py::test_custom_cached_disk test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio 'test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node]' test_filesystem_cache/test.py::test_system_sync_filesystem_cache test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_http_and_readonly/test.py::test_http_get_is_readonly test_http_native/test.py::test_http_native_returns_timezone test_inherit_multiple_profiles/test.py::test_combined_profile test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_jbod_ha/test.py::test_jbod_ha test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin test_keeper_block_acl/test.py::test_keeper_block_acl test_keeper_compression/test_with_compression.py::test_select test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks test_keeper_four_word_command/test_allow_list.py::test_allow_list test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password test_keeper_memory_soft_limit/test.py::test_soft_limit_create test_keeper_mntr_pressure/test.py::test_aggressive_mntr test_keeper_nodes_move/test.py::test_node_move test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout 'test_keeper_zookeeper_converter/test.py::test_acls[False]' 'test_keeper_zookeeper_converter/test.py::test_acls[True]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_smoke[False]' 'test_keeper_zookeeper_converter/test.py::test_smoke[True]' test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain]' test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.